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ABSTRACT 

The first goal of this study was to examine the impact of nine 
World Wide Web-based learning modules on learning as measured on both online 
module -quizzes and in-class exams. The modules were designed to support' 
fundamental concepts in entry-level college mathematics courses. The second 
and third goals of this study were to determine the learner characteristics 
and strategies that affect student performance on the nine Web-based learning 
modules. Data sources included: a demographic .survey; nine Web-based 
instructional module quizzes; three in-elass quizzes; three in-class, paper- 
based quiz surveys; an in-class final exam; an exit survey; and face-to-face 
interviews. Findings indicated that students who scored above 80% on the 
module quizzes also did better on in-class exams. Those who were self- 
motivated, focused, and self-disciplined had greater success in the online 
module environment than students_ who participated haphazardly or erratically 
in the modules.. It is concluded that Web-based modules support learning when 
used systematically by learners and that such modules extend the reach of the 
classroom teacher and reinforce classroom instruction. (Contains 26 
references.) (Author/MES) 
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Abstract: The first goal of this study was to examine the impact of nine Web-based learning 
modules on learning as measured on both online module quizzes and in-class exams. The modules 
were designed to support fundamental concepts in entry-level college mathematics courses. The 
second and third goals of this study were to determine the (a) learner characteristics and (b) 
strategies that affect student performance on the nine web-based learning modules. Findings 
indicated that students who scored above 80% on the module quizzes also did better on in-class 
exams. Those who were self-motivated, focused and self-disciplined had greater success in the 
online module environment than students who participated haphazardly or erratically in the 
modules. We conclude that Web-based modules support learning when used systematically by 
learners and that such modules extend the reach of the classroom teacher and reinforce classroom 
instruction. 



Objectives 
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Although web-based computer-assisted instruction is becoming openly accepted and used, there is little 
quantifiable data as to its effectiveness in helping students become successful learners. Given the money being 
allocated and commitments being made to the use of technology in homes and schools, it is incumbent upon 
educators to demonstrate that student use of technology can result in adequate or above average student success. In 
addition, researchers need to identify those learner characteristics and behaviors that foster quality learning and 
development in technologically-based environments so that educators can provide differential instruction to their 
students. 

This study examined the impact of nine web-based learning modules, designed to support fundamental 
concepts in entry-level college mathematics courses, on learning as measured on both online module quizzes and in- 
class exams in two university level mathematics courses. Concepts were presented in class and then students were 
expected to interact with the online modules to expand and clarify their in-class learning. The second and third goals 
of this study were to determine the learner characteristics and strategies that affected student performance on the 
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nine web-based learning modules as well as course test scores. Specifically, answers to the following questions were 

sought. 

1. How does the use of the nine web-based learning modules designed to support fundamental concepts in entry- 
level college mathematics courses impact student success on in-class tests and quizzes? 

2. What are some of the characteristics of learners who exhibit satisfactory performance on both the nine web- 
based learning modules and in-class tests and quizzes? 

3. What are some of the learning strategies of learners who exhibit satisfactory performance on both the nine web- 
based learning modules and in-class tests and quizzes? 

4. What are some of the characteristics of learners who exhibit unsatisfactory performance on both the nine web- 
based learning modules and in-class tests and quizzes? 

5. What are some of the learning strategies of learners who exhibit unsatisfactory performance on both the nine 
web-based learning modules and in-class tests and quizzes? 



Theoretical Framework 

The explosion of available online educational materials and increased access to technology and the Internet 
in classrooms across the country has shifted educators’ attentions to areas such as computer-assisted and online 
learning (Anglin & Morrison, 2000; Brown, 2000a; Brown, 2000b; Francek, 2000; Kessler, Rosenblad, & Shepard, 
1999; Liaw & Huang, 2000; Levinson & Surratt, 2000; Lyall & McNamara, 2000; Mioduser, Nachmias & Lahav, 
2000; Norman, 2000; Owston, 1997; Reiber, 1992; Spodick, 1995; Starr, 1996; Wagschal, 1998). With the ability to 
easily access online and interactive learning materials from any location via an Internet connection, parents, 
educators, and legislators are seeing the Internet as a cost effective way of leveling the learning environment for 
students across the country (Francek, 2000; Kessler, Rosenblad, & Shepard, 1999), as well as tailoring the learning 
environment for each learner (Haury, 1999; Keup 1998; MacKnight, 1998; Roth, 1999; Svetcov, 2000). 

Researchers and practitioners alike have expressed concern about whether students can learn as well in a 
web-based environment as they do in a face-to-face environment (Bonk & Dennen, 1999; Diaz & Cartnal, 1999; 
Leasure, Davis & Thievon, 2000; Sankaran, Sankaran & Bui, 2000; Zielinski, 2000). Studies have shown that 
learning can actually be enhanced using the World Wide Web. Ross and Schultz (1999) and Sankaran, Sankaran and 
Bui (2000) found that the use of innovative Web applications could address a wide range of diverse learning styles. 
Online conferencing gives students who might be too shy to speak up in a face-to-face setting a chance to voice their 
opinions and participate in discussions they might not normally participate in (Bonk & Dennen, 1999). Instead of 
being tied to a traditional classroom schedule, students are able to participate in Webb courses at a time and in a 
place that works best for them (Leasure, Davis & Thievon, 2000). 



Methodology 

This study examines the impact of nine JAVA -based computer modules in mathematics on learning as 
measured by quizzes and in-class exams in core curriculum, university level, Finite mathematics courses. The study 
also looks at learner characteristics, (e.g., background, course goals, etc.) and at strategies employed by students 
while working on the modules. This is a mixed methods study in that it features both qualitative and quantitative 
methods for data collection and analysis. 



Participants 

Subjects. The participants in this study included 147 students enrolled in two fall semester sections of MATH 
141(Business Mathematics I) and 191 students in two fall semester sections of MATH 166 (Topics in Contemporary 
Mathematics II). The total sample size for the study was 338. 

Courses. One of the selected MATH 141 classes will have access to the nine web-based instructional modules 
although participation will be optional for these students. The second Math 141 class will not have access to the 
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modules and will be used as a control group. For the MATH 166 courses, students will be required to complete the 
nine web-based instructional modules over the course of the semester, (approximately one per week). The four 
classes will create a continuum of module use going from nonuse to partial use to complete use and interaction. 
These classes were chosen for the study because of the close alignment of curriculum goals and course work with the 
scope and content of the modules. Although the two courses are designed for different populations, the content of 
both courses is virtually identical. Concepts, order in which material is presented, and amount of class time spent on 
each topic are the same. In addition, the instructors for each of the classes using the modules is a developer of the 
system, hence, they should be able to seamlessly incorporate the modules into the class curriculum. 



Procedures 

Access. Students gained access to the modules over the Internet. All students were required, as a part of their 
coursework in mathematics, to obtain a TAMU student computer access account. Modules were accessed through a 
secure password protected account. This means that only students enrolled in the specified sections were able to 
login to the modules through the campus open access computer labs or through personnel PCs. The math modules 
run on any operating system platform using a JAVA enabled browser (e.g. Netscape or Internet Explorer). 

Data Sources. Data Sources included (a) a demographic survey, (b) nine web-based instructional module quizzes, (c) 
three in-class quizzes, (d) three in-class, paper based quiz surveys, (e) an in-class final exam, (f) an exit survey, and 
(g) face-to-face interviews. 

At the beginning of the semester, students completed an online demographic survey ascertaining the 
gender, age, course expectations, and study habits of each participant. By being in a proctored room, the researcher 
assured that each student was completing the tests themselves without the help of study aids, calculators, or outside 
assistance or help. The demographic survey information was used to make sure that the four classes being studied 
were similar in makeup and overall ability. 

During the course of the semester, individual module login data, and quiz performance data on the nine 
web-based instructional modules were gathered and recorded into a secure database. Data related to the use of the 
modules — (a) number of attempts to take each quiz, (b) first score on each module quiz, (c) last score on each 
module quiz, and (d) highest score on each module quiz for each student, were gathered and sent to asecure 
database. 

As a normal part of expected course work, students completed three in-class quizzes. The quizzes took 
place approximately once a month, were created by each individual instructor and consisted of both multiple choice 
and workout problems . 

Students also were asked to complete a survey each time they took an in-class quiz, establishing study 
habits and student expectations. The survey specifically looked at how students used the modules as a part of their 
study and preparation for each major exam as well as their judgement of module effectiveness for facilitating their 
primary learning goals. 

At the end of the semester students took an in-class final exam. While all instructors’ exams were unique, 
they all contained a sampling of 4 or 5 identical questions aligned with the concepts covered both in text and in the 
online applets. 

All participating students will also be asked to complete an exit survey at the end of the semester, which 
will be given at the end of their final exam or during the study session just prior to their final exam. In addition to 
the eleven questions that they have been ranking on a Likert scale following each of the three in-class quizzes, this 
survey will also include fourteen additional open-ended questions. These additional questions will be optional for 
the students and will focus on the strategies students used during the course of the semester to complete the online 
modules. 

During the final exam, participants will be asked to complete one last survey. This survey well again ask 
students to identify their gender and class ranking as well their current and expected grades for the course. Students 
will then be asked to rank their feelings about the modules on a 1 to 100 point Likert Scale. 

Interviews at the end of the semester will be used to add description and validity to the information 
gathered through the quiz surveys. Following the third in-class quiz, students’ grades on module quizzes one 
through eight and grades on in-class quizzes one through three will be analyzed in order to place students within a 4 
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by 4 matrix. Three students from each matrix category will be randomly chosen for an interview. This should 
provide a list of twelve randomly selected students. Interviews will be conducted between the third exam and the 
end of the semester. 



Data Analyses 

By creating a mixed methods study, an attempt was made to not only to verify that there was some learning 
effect but also to identify some learner characteristics and strategies, which could interact with the modules to effect 
learning. The quantitative data, which included the scores on the nine web-based instructional module quizzes, 
scores three in-class quizzes, and the score on the in-class final exam, was the most effect way to look for a 
relationship between student achievement and the modules and to answer question one. By correlating scores on the 
quizzes included as a part of each of the nine-based modules and scores on the in-class exams and the final exam, as 
well as the final course grade a re lationship should become evident. 

Module quiz scores, in-class exam scores, and final course grades were used to answer question one, how 
did the use of the nine web-based learning modules designed to support fundamental concepts in entry-level college 
mathematics courses impact student success on in-class tests and quizzes. Module quiz scores, in-class exam scores 
and final course grades were analyzed to determine if modules had had an effect on learning in mathematics. 
Highest module quiz grades for each module quiz were averaged and then compared to average exam scores and the 
final course grade across groups: Module Use Group, Optional Module Use Group and the Control Group. The 
correlation coefficient is a direct measure of the relationship between variables. To measure the association between 
the modules, exam scores and final course grades, a Pearson product-moment correlation was run. To further 
compare the modules, exam scores and final course grades a Multivariate Analysis of Variance (MANOVA) was 
run. Scores on exams and the final course grade are the two dependent variables, and our hypotheses are that each 
will be positively affected by module use. When a statistical significance was found then Tukey's Post Hoc Test was 
run. A Tukey's Post Hoc Test will show which comparisons are significant when all possible pair wise comparisons 
are carried out. 

The qualitative date which included the demographic survey results, the three in-class, paper based quiz 
survey results, the exit survey results, and the twelve face-to-face interviews, most effectively looked for learner 
characteristics and strategies of students who use the web-based modules. By using the survey information and 
interviewing students face-to-face, there is a chance to identify some of those learner characteristics and strategies 
that most effect learning with the web-based modules. The interviews and observations made during class visits and 
discussions with each professor should provide thick description about what is actually happening with the students 
during actual module use. 

To answer questions two and three, what were some of the characteristics of learners who exhibited 
satisfactory performance on both the nine web-based learning modules and in-class tests and quizzes and what were 
some of the learning strategies of learners who exhibited satisfactory performance on both the nine web-based 
learning modules and in-class tests and quizzes, module quiz scores and learner characteristics as identified through 
the demographic survey were analyzed to determine if they could have had a confounding effect on learning 
attributed to module use. A statistical frequency was run on the demographic data and analyzed to determine 
differences between the groups that could have confounded effect results. While the demographic survey looked at 
multiple learner characteristics, variance was only found in gender, major, grade in high school algebra class, high 
school geometry class, student’s comfort with computers and average reported web browser use. Highest module 
quiz grades for each module quiz were averaged and then compared across learner characteristics. To further 
compare the module quiz scores and the demographic variables a Multivariate Analysis of Variance (MANOVA) 
was run comparing highest module quiz scores, final course grades and learner characteristics. If a statistical 
significance was found then Tukey's Post Hoc Test was run in order to determine which comparisons were 
significant. 

Interviews, the three in-class exam surveys, the final exam survey, the post final exam survey, and average 
module quiz scores and the average number of module quiz attempts for each of the nine modules were analyzed in 
an attempt to answer questions four and five, what were some of the characteristics of learners who exhibited 
unsatisfactory performance on both the nine web-based learning modules and in-class tests and quizzes and what 
were some of the learning strategies of learners who exhibited unsatisfactory performance on both the nine web- 
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based learning nodules and in-class tests and quizzes. Module data was analyzed to determine what learner 
strategies for completing the modules, in regards to the number of attempts made to complete each module quiz and 
the scores received on each attempted module quiz developed over the course of the semester. How much time 
students spent in each module, how students performed on module quizzes and how students approached module 
quizzes was of particular interest in establishing learner strategies in using the modules. 

An analysis of the three in-class exam surveys was done to look for emerging learner strategies. 
Specifically, a statistical frequency was run comparing in-class survey one, two, three, and the final exam survey 
results. The frequencies were analyzed to determine changing trends in study habits, allotted study times, and 
perceived effectiveness of these activities. 

A content analysis was used to identify emerging topics and themes regarding students’ learning strategies 
described by students during interviews and on the post final exam surveys. Participants were assigned a pseudonym 
and a number, which corresponded their placement within the interview selection matrix. Transcripts and survey 
questions were analyzed and separated into meaningful units. Cards were created for each unit of meaning and each 
card was assigned a string of numbers. Cards were coded for card number, type of data (interview or survey), 
interviewee, date of interview, and interview matrix location if applicable. For example: 12. 1. SS. 12/02/01. 1/1 would 
indicate that card 12, was part of an interview with Sam Spade conducted on December 2, 2001 and he was a part of 
the low number attempts and high module quiz grade group. This form of coding was used to create an audit trail. 

Interviews and surveys provided invaluable sources of data. Themes and topics as well as supporting 
statements were determined by multiple sources including these interviews and post final surveys, as well as the in- 
class surveys and observations made throughout the duration of the study. 



Results 

Findings indicated that the web-based modules contributed positively to students performance in the course. 
Average module quiz scores, adjusted exam scores, and final course average were highly correlated. Students who 
scored above 80% on the module quizzes also did better on in-class exams and final course average. Based upon 
qualitative data, including ongoing in-class surveys and follow-up interviews we found that those who were self- 
motivated, focused and self-disciplined had greater success in the online module environment than students who 
participated haphazardly or erratically in the modules. Students who had a plan for completing the module activities 
and quizzes scored higher on the quizzes and in-class exams. Many students who skipped through the module 
activities as quickly as possible, without reading the information presented or working through the interactive 
activities, eventually did well on the module quizzes. These learners often made ten or more attempts at the quizzes 
before achieving a score of 80% or better, but they did not do as well on the in-class exams. Planning, applying the 
modules repetitively and thoughtfully, and consciously making connections between course activities and module 
activities weie strategies that positively impacted module effectiveness. Based on these findings, we conclude that 
Web-based modules supported learning when used systematically by learners. Conversely, students who did not 
plan, repetitively engage in the modules, or make connections did not successfully use the modules for learning. 



Significance of the Study 

The Internet can be a powerful and useful tool. Education can no longer ignore the Internet's potential for 
the classroom and learning but neither can education abandon its need for standards and accountability. This study 
focuses on the connection between web-based learning modules and their impact on student success as well as 
identifying some of the characteristics of learners who succeed or do not succeed in this environment. The insights 
gained here can help to provide a basis for designing additional web-based, modular environments in which students 
can find learning success can be used to identify students who may require preliminary training in how to leam 
online. In addition, effective strategies identified in this research can be the basis of such preliminary training. 
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